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Quality  Function  Deployment  model.  Its  structure  facilitates  communication,  planning, 
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GUIDE 


1.  INTRODUCTION 

TRAC  conducts  trade  space  analyses  as  part  of  our  analytical  support  to  inform  Army  decision 
makers.  Other  industries  might  consider  these  ‘trades’  as  design  decisions  or  design  tradeoffs 
(Decision  Driven  Solutions,  2008).  Regardless  of  their  title,  the  purpose  is  the  same:  facilitate 
decisions.  The  intent  of  this  document  is  to  provide  an  overview  of  trade  space  analysis;  to 
include  terminology,  key  principles,  characteristics,  and  challenges.  The  objective  is  to  keep 
these  descriptions  generalized  to  emphasize  their  application  regardless  of  the  analysis’  focus. 
This  is  a  different  approach. 

This  document  is  a  descriptive  guide1.  It  is  intended  for  those  analysts  conducting  or  supporting 
a  trade  space  analysis  or  other  trades-related  study.  This  guide  may  provide  some  utility  to  those 
overseeing  the  execution  of  the  study  as  a  primer  for  trades  analysis  study  updates  or 
methodology  reviews.  This  guide  will  not  to  tell  you  how  to  do  trade  space  analysis,  but 
describe  the  various  components  of  what  is  considered  trade  space  analysis.  The  intent  is,  by 
describing  these  components  and  associated  attributes,  that  it  helps  generate  ideas,  facilitate 
planning  and  methodology  selection,  etc.  There  are  links  to  a  number  of  resources  throughout; 
these  are  provided  to  support  the  conduct  of  a  thorough  literature  review  and  to  reduce  the  tune 
spent  searching  for  potential  references. 

To  formally  introduce  the  topic,  let’s  use  a  familiar  example  of  trade 
space  analysis.  Consider  the  elements  of  project  planning  depicted  in 
Figure  1 :  Project  Management  Triangle,  (aka  the  “Good,  Fast,  Cheap” 
model).  This  is  a  veiy  simple  example  of  trades  but  the  same  elements 
hold  true  for  more  complex  examples. 

Studies  can  be  completed  with  a  high  level  of  quality,  using  extensively 
developed  scenarios,  high  resolution  modeling,  detailed  data  sets,  with 
multiple  analysts’  expertise  and  methodologies  applied.  Studies  can  be 
completed  quickly  given  the  requisite  resources  (i.e.  personnel, 
computational  power,  accessibility  to  data,  etc).  All  of  these  descriptors 
of  quality  and  timeliness  are  associated  with  a  cost.  Without  fail, 
limitations  to  these  financial  or  other  finite  resources  force  compromise. 

So,  what  gives?  This  shapes  trade  space. 

Each  of  the  relationships  between  the  elements  in  the  triangle  can  be  described.  Some 
relationships  are  concrete,  more  easily  measurable  while  others  are  more  ambiguous.  Different 
priorities  amongst  the  elements  will  affect  the  outcome.  The  purpose  of  trade  space  analysis  is  to 
identify  the  significance  of  those  relationships  and  quantify  them  in  a  meaningful  way. 


A 


Cost 

Figure  1:  Project  Management 
Triangle 


1  This  document  is  supplemented  by  the  contents  of  the  Trade  Space  Community  of  Practice  page.  On  the 
page:  articles,  briefs,  links  to  TRAC  products,  and  areas  for  analysts  to  collaborate  on  the  topic  and  share  their 
experience.  Tlus  is  intended  to  be  a  living  document.  Please  feel  free  to  reference  any  part  of  this  guide  as  a 
discussion  topic  for  recommendations  for  further  revisions  or  topics  to  cover. 
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2.  DEFINITION  OF  TRADE  SPACE 


In  literature  and  TRAC  work  there  are  a  number  of  instances  where  this  term  has  different 
meanings. 

Tradespace  (synonymous  with  trade  space,  Trade  Space,  trade-space ):  a  combination  of  the 
words  “tradeoff’  and  “playspace”  2  (the  mathematical  optimal  boundary,  aka,  Paretto  Frontier). 

Tradeoffs  (synonymous  with  trades,  trade-offs,  trade  offs):  (noun)  alternative  key  objectives  all  of 
which  cannot  be  attained  together  in  a  decision,  design,  or  project  and  their  associated  benefits 
and  opportunity  costs.  (Business  Dictionary) 

Tradeoff:  (verb)  reducing  or  forgoing  one  or  more  desirable  outcomes  in  exchange  for  increasing 
or  obtaining  other  desirable  outcomes  in  order  to  maximize  the  total  return  or  effectiveness  under 
given  circumstances.  (Business  Dictionary) 

Here  are  two  separate  academic  articles’  definitions  of  the  same  term  in  a  military  context: 

Trade  space:  “the  set  of  program  and  system  parameters,  attributes,  and  characteristics  required 
to  satisfy  performance  standards”  (Brantley,  MAJ  USA,  McFadden  LTC  USA,  &  Davis  LTC 
USA  (Ret),  Winter  2002). 

Trade  space:  “is  a  wide  spectrum  of  potential  outcomes  spanned  over  a  set  of  operational  and 
system  parameters,  attributes  and  characteristics  to  provide  possible  design  options  and  satisfy 
operational  expectations  and  system  performance  criteria.”  (Hong,  Wee,  &  Kiat). 

Is  it  the  “set”  or  is  it  the  “potential  outcomes  of  the  set”?  These  definitions  are  specifically 
related  to  system  design.  As  the  example  in  the  introduction  suggests,  there  are  other  trades  and 
spaces  to  consider.  Since  TRAC  studies  are  not  uniform,  our  definitions  must  be  generalized 
enough  to  encompass  the  variation: 

Trade:  an  attribute  or  characteristic  [of  a  design,  decision,  etc.]  with  associated  benefits  and 
opportunity  costs  which  may  be  exchanged  in  part  or  totality.  Related  term(s):  parameters,  input 
variables,  objectives,  constraints. 

Trade  space:  the  bounded  area  which  considers  the  range  of  possible  values  (inherent  or  applied) 
for  any  number  of  attributes  and  characteristics,  the  relationships  between  them,  and  impacts  on 
potential  (design,  decision,  operational)  outcomes. 

Trade  space  analysis 3:  the  search  of  the  bounded  space  to  highlight  the  relationships  between 
trades,  their  values  (inherent  or  applied),  and  outcome  objectives  to  inform  decision  makers. 
Related  terms(s):  multi-attribute,  interdependent,  decision  makers. 


2  (1)  Paretto  Frontier  is  referred  to  as  “playspace”  in  several  instances  in  the  literature.  (2)  Interestingly, 

according  to  Wikipedia,  the  term  tradespace  has  application  as  an  organizational  process  management  term;  a 
collection  of  processes  spanning  multiple  organizations.  Consider  the  use  of  this  definition  in  the  context  of  Force 
Design  or  other  organizational  analyses  (i.e.  identifying  organizational  efficiencies). 
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3.  WHEN  DOES  TRAC  CONDUCT  TRADE  SPACE  ANALYSIS? 


The  DoD  analytic  communities  recognize  that  there  are  challenges  and  shortcomings  within  DoD 
decision  making  processes  (program  overruns,  delays,  etc.)-  The  basic  source  of  these 
challenges  is  the  stove  piped  nature  of  decision  making  processes.  In  the  corporate  world,  all 
decisions  at  every  level  are  aligned  with  a  corporate  strategy.  Analytic  approaches  are  usually 
finance-centric,  marketing,  or  economics  driven.  Figure  2  shows  that  it  is  not  necessarily  that 
simple  in  the  analytical  environment  in  which  we  work. 


Private  Sector 


...If  you  make  a  decision  to  invest  capital,  particularly  a  sizeable  decision  to  invest  capital,  that  goes  all  the 
way  to  the  chairman  and  probably  to  the  board  if  it’s  a  reasonable  amount. .  .  .  Everyone  within  the 
company — directors  of  the  manufacturing,  marketing,  sales,  finance  and  other  departments — - 

decision  and  commits  .  .* 


Integrated  management  and  communication  tools. 
Metrics  and  measures  traceable  to  bottom  line. 


Integrated  and  shared  vision;  align  short  term  and 
long  term  goals. 


Government  /  DoD 


.  .  .a  tremendous  number  of  stakeholders  often  work  toward  contradictory  goals,  and  year-to- 
year  budget  fluctuations  can  derail  an  initiative  before  it's  able  to  bear  fruit.  .  .* 


Multiple  decision  makers  and  communication  tools.*  Stakeholders  not  necessarily  aligned  with 

, ,  .....  decisionmakers 

Effectiveness  is  measured  by  capability. 


TRAC 


Must  conduct  the  studies  that  inform  key  decisions 
made  by  TRADOC,  Army,  and  Joint  Leaders. 


AoA  /  C-BA 
Force  Design 


Portfolio  Review 
Decision  Support 


Develop  and  inform  capability  measures. 

Support  development  and  alignment  of 
requirements. 


Coordinate  amongst  multiple  stakeholders  and 
decision  makers. 


*Ken  Krieg  -  Former  Undersecretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 


Extended  time  periods,  multiple  decision  makers  and  stakeholders,  stove-piped 
information  availability  create  a  challenging  environment  to  conduct  trades  analysis. 


Figure  2:  The  challenge  associated  with  trade  space  analysis 


TRAC  supports  a  number  of  different  types  of  studies3 4.  The  introduction  alluded  to  the  fact 
there  is  really  no  limit  to  situations  were  trades  occur.  Trades  are  most  commonly  referred  to  in 
a  capability  based  context,  either  supporting  an  analysis  of  alternatives  (AoA)  or  the  initial 
determination  of  capability  requirements  type  studies.  Each  decision  point  within  the  JCIDs 
acquisition  process  presents  an  opportunity  to  conduct  a  trades-related  assessment.  However, 


3  One  observation  from  this  research  was  that  there  is  a  slight,  but  relevant,  difference  between  trade  space 
analysis  and  tradeoff  analysis.  Trade  space  analysis  is  the  umbrella  term.  However,  based  on  the  literature  review, 
tradeoff  analysis  insinuates  a  certain  level  of  “concrete-ness”  of  alternative  definitions,  attributes,  capabilities,  and 
desired  outcomes. 

4  The  project  report  (TRAC-M-TR-15-028)  associated  with  this  guide  contains  a  list  of  example  TRAC  studies 
along  with  descriptions  of  their  trades  analysis  objectives,  methodologies,  and  trades  considered.  The  Trade  Space 
Community  site  links  to  a  majority  of  these  project  reports. 


3 


there  are  budget,  planning  and  composition,  and  mix  problems  where  any  combination  of 
capability,  cost,  and  value  attributes  are  traded. 

4.  ELEMENTS  OF  TRADE  SPACE  ANALYSIS 


If  you  keep  the  elements  of  decision  making5  in  mind  when  you  think  about  the  trade  space 
analysis  process,  there  is  nothing  too  special  or  complicated:  define  criteria,  define  alternatives, 
evaluate  alternatives,  and  select/commit  to  a  best-fit  alternative  for  implementation.  “The  trade 
space  exploration  methods,  processes,  and  tools  should  enable  deeper  consideration  of  system 
design  alternatives  while  keeping  the  space  as  open  as  possible  to  address  resiliency  and 
robustness  to  changing  conditions  and  constraints.”  (Spero,  Avera,  Valdez,  &  Goerger,  2014). 
Figure  3,  from  the  referenced  article  above  shows  the  types  of  information  that  is  required  to 
support  decision  making  across  the  lifecycle  of  a  particular  system.  It  takes  into  account  the 
number  of  perspectives,  environmental  constraints,  fiscal  decision  types,  etc.  that  must  be 
considered.  This  graphic  emphasizes  the  complexity  of  data  involved  when  conducting  trades 
analyses. 


Figure  3:  Tradespace  Frontier 


Not  all  of  TRAC  trades  analyses  focus  on  system  design.  However,  in  the  past,  a  greater 
proportion  of  them  do.  The  article  “Tradespace  Exploration  for  the  Engineering  of  Resilient 
Systems”6  succinctly  presents  12  best  common  steps  of  conducting  trade  space  exploration. 
Even  though  the  list  was  written  specifically  about  the  conduct  of  trades  analysis  at  the 
engineering  design  level,  the  same  practices  apply: 

Table  1:  Best  Common  Practice  Steps 

1.  Determine  mission  scenario(s)  and  their  requirements,  and  keep  them  open  as  long  as  possible. 


5  Even  decisions  have  ‘ility’  attributes  (e.g.)  flexibility,  adaptability,  responsiveness.  These  are  not  commonly 
assessed  /  considered  as  part  of  TRAC  trade  space  analysis. 


6  The  article  is  on  the  community  page. 
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2.  Identify  set  of  operational  performance  characteristics  and  high  level  system  design  variables  that 
impact  operational  requirements. 

3.  Apply  operational  engagement  models  against  various  mission  scenarios  and  threats  to  identify 
requirements,  MOP,  MOE,  and  other  performance  metrics. 

4.  Expert  knowledge  teams  determine  values  of  measures  for  given  mission  scenarios  and 
requirements. 

5.  Break  down  stakeholder  values  into  roles,  attributes,  and  specific  tasks. 

6.  Generate  alternatives  that  meet  requirements  and  constraints,  and  map  stakeholder  values  to 
system  design  variables  using  scalable  multi-physics  based  modeling  design  tools. 

7.  Create  reduced-order  surrogate  models  to  show  iterative  ability  of  adjusting  scenarios  and 
requirements  to  physical  feasibility. 

8.  Qualitatively  or  quantitatively  rank  how  alternatives  meet  measures. 

9.  Perform  a  LCC  estimate  and  lifecycle  schedule  analysis  of  the  system. 

10.  Perform  an  optimization  study  to  determine  the  optimum  feasible  space  that  meets  all  constraints 
and  for  each  course  of  action. 

1 1 .  Determine  courses  of  action  based  on  optimal  feasible  space  and  perform  post-analysis  studies 
(operational  impact  and  gap  analyses). 

12.  Perform  case  studies  to  test  for  robustness  and  to  make  sure  that  the  alternative  solutions  are 
resilient  in  changing  operational  environments. 

The  following  sub-sections  are  broken  out  into  more  general  steps  of  conducting  a  study  in  order 
to  highlight  attributes  common  to  any  type  of  problem.  The  related  steps  from  Table  1  will  be  in 
the  brackets  at  the  beginning  of  each  section. 

4.1  Define  the  problem. 

(1)  Any  analysis  effort  starts  with  problem  definition.  The  output  from  the  initial  problem 
decomposition  or  “Front  End  Analysis”  and  development  of  measurement  space  should  provide 
the  majority  of  starting  conditions  and  requirements  for  any  trade  space  analysis. 

4.2  Define  (identify)  trades. 

(2)  Through  the  systematic  decomposition  of  the  problem,  the  attributes  and  characteristics 
become  clear.  There  are  a  number  of  activities  that  provide  that  clarity:  workshops  (e.g. 
Measurement  Space  Drill  or  other  workshops),  stakeholder  analysis,  M&S,  and  document 
reviews.  Depending  on  the  problem  type,  there  will  be  requirements  documents  (e.g.  ICD  or 
CDD)  that  will  help  define  attributes  and  characteristics.  It  “is  important  to  qualitatively  and,  if 
possible,  quantitatively  describe  the  interactions  between  the  trade  space  entities.  These 
interactions  highlight  stakeholders  that  are  impacted  by  decisions  for  each  of  the  trade  space 
entities  and  identify  the  critical  interactions  that  will  require  data  collection,  modeling,  or 
simulation  to  understand  the  relationship.  This  process  can  also  reduce  the  problem  complexity 
using  known  relationships  constraints  and  thresholds.”  (Brantley,  MAJ  USA,  McFadden  LTC 
USA,  &  Davis  LTC  USA  (Ret),  Winter  2002).  There  are  a  number  of  descriptors  that  are 
related,  but  may  not  affect  the  outcome. 
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Note:  Trades  may  originate  from  any  domain  within  the  DOTMLPF-P  spectrum.  Each  domain 
is  considered  a  separate  bin  of  different  kinds  of  products  and  services  necessary  to  conduct 
business.  A  gap  or  capability  requirement  might  have  alternative  solutions  which  originate  from 
specific  or  across  bins.  It  is  usually  assumed  that  trades  assessments  are  confined  within  the 
DOMTLPF-P  domains  and  not  necessarily  between  domains.7 

4.3  Describe  trades. 

Recall  from  the  trade  space  definitions  that  attributes  may  have  inherent  and  /  or  applied  values. 

(3)  Inherent  Value.  These  describe  the  attributes  directly.  Each  of  these  descriptors  has  their 
own  unit  of  measure.  Regardless  of  the  type  of  value  or  the  problem  scope,  some  attributes 
already  have  threshold  or  objective  levels  defined.  These  values  can  be  ridged  objectives  and 
constraints  that  must  be  accounted  for,  while  others  are  more  flexible  or  elastic.  Based  on  all  of 
the  descriptors,  these  attributes  become  the  parameters  or  input  variables,  objectives,  and 
constraints,  areas  for  sensitivity  analysis.  Values  should  articulate  provided  benefit  (e.g. 
performance  effectiveness  or  efficiency)  and  operational  costs  (e.g.  price  tag,  risk,  etc.) 
associated  with  the  element.  From  the  project  management  example;  time  can  be  measured  in 
any  increment  or  as  man  hours.  Whereas,  cost  can  also  be  measured  in  terms  of  man  hours  or 
dollars,  or  other  resource  costs,  like  computational  power  units  (e.g.  number  of  machines). 

(4/5)  Applied  Value.  These  include  stakeholder  and  decision  maker  priorities  and  /  or  the 
attributes  and  characteristics  they  might  value  over  others.  Again,  from  our  example,  let’s  say 
that  decision  makers  are  more  concerned  with  maintaining  quality,  within  a  restricted  time  frame 
than  cost.  The  question  becomes,  how  much  more? 

TRAC  has  several  guides  to  elicitation  that  may  help:  SME  Elicitation  Community  Page,  Survey 
Code  of  Practice,  SME  Elicitation  Technical  Report  I  and  II.  Related  topics  are:  Stakeholder 
analysis,  value  focused  thinking,  decision  analysis  methods  such  as  qualitative  value  modeling, 
multi-attribute  decision  making  techniques.  TRAC  has  a  Multi- Attribute  Decision  Making  Code 
of  Best  Practice8  that  provides  an  overview  of  methodologies  and  considerations. 

4.4  Assess  trades. 

At  this  point,  we  articulate  relationships  between  trades.  The  curves  in  the  graphic  below 
describe  the  relationships  between  the  attributes  in  the  project  management  triangle. 


7  All  of  DOTMLPF-P  domains  are  tied  to  Doctrine.  Any  change  in  the  other  domains  will  ultimately  have  an 
effect  on  Doctrine  and  vice  versa.  (Alfred,  2007)  The  common  currency  across  domains  is  usually  dollars;  however, 
there  are  certain  domains  where  comparable  benefit  /  opportunity  cost  related  measures  are  difficult  to  articulate. 

°  The  TRAC  SharePoint  main  page  has  a  link  to  all  Codes  of  Best  Practices. 
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Figure  4:  Relationships  between  Project  Management  Trades 


Scope  (upper  left):  given  enough  resources  you  can  achieve  almost  anything. 

Quality  (upper  right):  there  is  a  certain  level  of  quality  that  will  be  sufficient.  However, 
there  is  a  certain  point  where  it  doesn’t  matter  how  much  more  time  or  money  you  throw 
at  it,  it  may  not  get  any  better. 

Cost  (bottom) 9:  If  it  needs  to  be  done  faster,  it  will  be  more  expensive.  However,  there 
is  a  threshold  where  it  will  compromise  the  integrity  of  the  project  if  you  spend  less  than 
x  time. 


These  can  be  generated  through  heuristics;  however,  there  are  a  number  of  MMT  that  apply  to 
this  problem.  Much  of  the  literature  has  extensive  detail  about  the  input  and  outputs  associated 
with  trades  analysis  and  little  detail  about  the  actual  methodologies  applied  to  exercise  the  data. 
The  associated  M&S  effort  (i.e.  venues,  methodologies,  tools,  etc.)  is  unique  to  each  set  of 
parameters  being  assessed.  TRAC  has  an  OR  Guide  that  provides  a  good  overview  of  a  number 
of  methods  for  grouping  and  comparing  multivariate  data  output  from  the  M&S. 

4.1.1  System  Attributes  and  Capability  Requirements  Trades 

(6,  7,  8,  9,  12)  Across  literature  these  system  related  trade  space  analysis  focus  on  performance, 
capability,  and  utility  values.  These  are  most  commonly  characterized  by  cost,  risk, 
perfonnance,  and  schedule  trades  in  TRAC  work. 

To  facilitate  this  portion  of  the  discussion,  consider  using  the  structure  of  the  Quality  Function 
Deployment  (QFD)  model  as  a  framework  for  building  out  a  trades  assessment.  It  is  part  of  the 
Six  Sigma  “Define  Phase.”  QFD  is  “a  structured  method  in  which  customer  requirements  are 
translated  into  appropriate  technical  requirements  for  each  stage  of  product  development  and 
production”.  The  primary  components  are:  defined  capability  (what)  requirements  (e.g. 
effectiveness  against  target  type  x  (lethality),  mission  duration  and  capacity  for  follow-on  tasks 


9  As  an  example  of  an  advanced  method  for  conducting  trades  analysis,  there  is  a  paper  related  to  applying  a 
genetic  algorithm  to  a  time-cost  trade  off  resource  planning  problem  located  on  the  Community  Site. 
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(efficiency),  protection  against  IEDs  (mobility),  etc.),  prioritized  rating  of  those  capabilities, 
perfonnance  characteristics  that  enable  capabilities  {how),  the  codified  interrelationships 
between  the  capabilities  and  performance  characteristics  along  with  the  tradeoffs  between 
performance  characteristics,  target  values  (i.e.  performance  (e.g.  speed,  probabilities  of  kill, 
capacity),  risk,  cost,  etc),  and  comparative  areas  for  why  the  requirement  exists.  Figure  4  maps 
the  essence  of  the  best  common  steps  to  components  of  the  model.  See  APPENDIX  C.  Q  for  a 
notional  example  using  LRPF-like  attributes. 
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Figure  5:  Quality  Function  Deployment 

The  house  of  quality  identifies  two  types  of  relationships  to  describe  through  analysis.  These  are 
the  relationships  between  capability  and  performance  inherent  values,  as  stated  above,  as  well  as 
the  tradeoffs  between  performance  attributes.  They  may  be  explored  qualitatively;  however, 
quantitative  analysis  is  usually  preferred.  Step  seven  in  the  “Best  Common  Practice  Steps” 
articulates  the  creation  of  “reduced-order  surrogate  models10  to  show  iterative  ability  of 
adjusting  scenarios  and  requirements  to  physical  feasibility”.  These  are  referred  to  as  response 
surface  models  or  meta-models.  They  mimic  the  behavior  of  the  simulation  as  closely  as 
possible  but  focus  primarily  on  the  input-output  relationship  to  be  able  to  handle  more 
exploration  with  less  computational  resources  required  in  more  high  fidelity  modeling.  These 
are  most  commonly  employed  in  engineering  design;  however,  there  is  application  in  a  number 
of  areas  were  simulation/experimentation  is  expensive.  If  only  one  design  variable  is  involved, 
the  process  is  called  curve-fitting. 


10  One  example  is  the  Relational  Oriented  Systems  Analysis  Engineering  Technology  Tradeoff  Assessment 
(ROSETTA)  Initiative  by  the  Aerospace  Systems  Design  Laboratory.  This  methodology  uses  qualitative  SME  data 
built  as  a  QFD  model  with  model  and  simulation  data  captured  using  response  surface  method  (RSM)  /  response 
surface  equations  (RSEs)  as  surrogate  models.  It  uses  the  RSEs  with  Monte  Carlo  simulation  to  quantitatively 
explore  changes  across  the  surfaces  to  develop  greater  understanding  of  the  relationships  represented  in  the  QFD. 
(ROSETTA,  2015) 
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As  a  side  note,  Multi-objective  decision  analysis  techniques  can  be  very  effective  for 
understanding  the  relationships  at  the  attribute  level  between  what  is  desired  and  what  is 
affordable.  Recently,  the  Brownout  Rotorcraft  Enhancement  Sensor  (BORES)  AoA  team  used  a 
related  approach.  The  AoA  supports  the  establishment  of  a  baseline  degraded  visual 
environment  capability  and  infonn  a  MSA  DAB  (TEDS).  The  Air  Force  has  actually  developed 
a  MOD  A  guide  to  facilitate  their  capability  cost  analysis11. 

4.1.2  Budget  /  Planning  /  Composition  Mix  Trades 

(10,  11)  These  assessments  can  be  described  as  a  subset  of  the  “Best  Common  Practice  Steps”. 
These  are  characterized  by  any  number  of  attributes  (cost,  performance  measures)  usually 
associated  with  applied  values.  These  analyses  can  be  heavily  influenced  or  driven  by  priorities, 
scheduling  constraints,  and  numerous  combinatorial  considerations.  TRAC  studies  successfully 
employ  math  programming  methods  to  such  problems.  Military  application  of  optimization 
methods  to  acquisition  planning,  BRAC  closures,  quantity  mixes  is  well  documented12.  The 
following  example13  describes  a  simple  knapsack  problem  considering  all  the  elements  we  have 
been  discussing.  It  includes  characteristic  “levers”  enabling  exploration  of  the  trade  space: 

The  Squad:  Foundation  of  the  Decision  Force  (SFDF)  Model  that  supported  a  study  of 
the  same  name.  It  was  a  fairly  simple  knap  sack  problem,  fueled  by  SME  established 
utility  values  for  capabilities,  cost,  gap  priorities,  and  a  number  of  “levers”  on  key 
attributes  to  run  the  model.  It  could  be  run  in  two  forms:  (1)  with  an  upper  and  lower 
limit  budget  and  a  defined  increment  and  (2)  with  a  fixed  budget.  Its  output  was  either 
the  top  capability  package  for  each  budget  amount  or  the  x  number  of  top  mixes  at  a 
fixed  budget. 

4.5  Communicate  trades. 

(8)  The  basic  element  relevant  to  trades  analysis  is  how  to  display  multivariate  data.  Seeing  and 
understanding  the  interaction  between  driving  factors  or  the  elements  more  highly  valued  is 
essential.  These  parameters  become  the  axis  or  levers  that  displays  should  revolve  around. 
Presenting  multiple  dimensions  of  data  in  a  single  display  is  an  art.  Basic  principles  are:  the  use 
of  color,  shape,  and  size.  There  are  a  number  of  chart  types  that  are  not  readily  available  in 
Excel,  but  available  as  online  chart  templates  or  as  R  packages14.  Textbooks  on  Multivariate 

1 1  This  information  was  shared  via  an  email  dated  20  April  15  from  Matthew  Boetig.  The  guide  is  in  DRAFT 
form  and  available  to  review  on  the  Community  Page. 

12  See  the  article  on  Optimizing  Military  Capital  Planning  for  an  overview  of  a  number  of  related  problem  types 
and  insights  on  formulation  development. 

1 3  The  TRAC  Math  Programming  Community  page  has  a  number  of  examples. 

14  TRAC’s  R  user  community  is  growing.  As  an  example,  the  TRADOC  Accessions  OPT  study  team  at 
TRAC-FLVN  is  using  GoogleVis  to  display  some  of  their  data.  They  are  using  it  as  an  exploratory  tool,  but  it  has 
benefit  as  a  communication  tool  as  well.  GoogleVis  is  an  R  package  that  provides  an  interface  between  R  and 
google  charts  API.  It  allows  users  to  use  charts  like:  Motion  Charts,  Annotated  Time  Lines,  Maps,  Geo  Maps,  Geo 
Charts,  Intensity  Maps,  Tables,  Gauges,  Tree  Maps,  further  Line,  Bar,  Bubble,  Column,  Area,  Stepped  Area, 

Combo,  Scatter,  Candlestick,  Pie,  Sankey,  Annotation,  Histogram,  Timeline,  Calendar  and  Org  Charts. 
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Analysis  are  obviously  a  good  resource  for  ideas.  Section  3.4  in  Rencher  &  Christensen’s  Text: 
Methods  of  Multivariate  Analysis  outlines  several  methods  for  plotting  three  or  more  variables  in 
a  two  dimensional  space.  Additionally,  data  presentation  texts  like  those  by  Edward  Tufte 
describe  techniques  to  try.  One  technique  he  recommends  is  to  use  small  multiples15  to  display 
effects  of  changing  variables. 

The  world  of  “what  if’  dictates  even  more  finesse.  The  use  of  decision  support  tools  such  as 
“dashboards”  have  become  common  practice.16  Graphs  show  the  relationships  between 
effectiveness  measures,  physical  performance  measures,  and  cost  and  constraints.  ‘Levers’  are 
on  critical  attributes.  These  facilitate  the  visualization  of  multiple  relationships  simultaneously; 
highlighting  interactions  in  a  meaningful  way;  identifying  data  gaps  and  outliers  and  correlations 
dynamically  based  on  decision  maker  driven  areas  of  interest.  Imagine  supplementing  briefing 
results  using  something  like  this.  These  are  built  using  surrogated  meta-models,  like  those 
mentioned  in  the  previous  section. 

5.  CHALLENGES 

Each  of  the  following  sub-sections  relates  to  potential  challenges  related  to  the  process  of 
conducting  trade  space  analysis.  The  majority  of  these  were  collected  through  discussions  with 
analysts  who  had  participated  in  some  of  the  trades  work  reviewed  for  this  study.  The  intent  of 
this  section  is  to  describe  the  nature  of  the  challenge  or  present  an  alternative  approach  to 
meeting  the  implied  challenge. 

5.1  Identifying  Trades. 

Alternative  focused  thinking  vs.  value  focused  thinking.  A  major  advantage  of  alternative- 
focused  thinking  is  that  the  analysis  focuses  directly  on  the  alternatives  of  interest  to  those  who 
put  forth  the  set  of  alternatives;  however,  this  assumes  that  the  overall  intent  can  be  met  with 
these  alternatives.  Value-focused  thinking  presents  an  approach  to  expand  the  scope  of  potential 
alternatives.  It  starts  with  values,  uses  values  to  generate  alternatives,  and  then  uses  values  to 
evaluate  those  alternatives  (usually  through  a  multiple  objective  decision  analysis 
technique)(Loerch  &  Rainey,  2007).  A  value-focused  thinking  approach  may  help  identify 
additional  areas  for  trades. 

5.2  Constructing  the  Trade  Space. 

Trade  space  links  requirement  and  capability  attributes.  TRAC  analysis  supports  requirements 
determination  during  Pre-MSA  work  through  the  entire  life  cycle  of  a  program.  For  example, 
The  Task  Force  120  GCV  analysis  work  focused  on  determining  the  initial  requirements  for  the 

15  Small  multiples  show  the  same  combination  of  variables,  indexed  by  changes  in  another  variable. 

“...showing  shifts  in  the  relationship  between  variables  as  the  index  variable  changes  (in  turn  revealing  interaction 
or  multiplicative  effects).”  (Tufte,  2001) 

16  There  are  a  number  of  examples  of  trade  space  analysis  and  related  decision  support  tools  that  have  been 
designed  across  the  DoD.  These  examples  span  a  number  of  decision  types,  maturity  of  system  development,  etc. 
See  APPENDIX  A.  DoD  EXEMPLARS  for  a  brief  description  of  a  few  of  them. 
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GCV  program  using  a  number  of  qualitative  value  modeling  approaches,  followed  by  MSA  and 
MSB  work  that  leveraged  more  and  more  quantitative  M&S  work;  the  advantage  being  the 
pedigree  of  linkage  back  to  initial  requirements.  Not  all  studies  are  so  lucky.  For  example,  the 
Bradley  ECP  C-BA  had  to  link  each  technology  they  assessed  as  potential  upgrade  back  to  a 
requirement  to  begin  their  analysis. 

5.3  Defining  input  variables. 

Tight  study  schedules  and  the  murkiness  of  early  project  phases,  present  a  challenge  to  not  only 
identifying  trades  areas,  but  identifying  data  elements  that  will  be  available  to  inform  analysis. 
LRPF  trades  team  mentioned  that  their  planning  was  scoped  by  the  tools  that  were  identified  as 
data  sources  to  collect  data  to  inform  metrics  /  measures  instead  of  the  other  way  around. 

5.4  Assess  trades. 

The  art  and  science  of  trade  space  identifies,  qualifies,  and  quantifies  the  relationships  between 
factors,  levels,  inputs,  outcomes,  etc.  There  is  usually  a  necessity  to  use  numerous,  and 
potentially,  stove-piped  tools.  This  presents  a  challenge  in  identifying  and  communicating 
relationships  between  findings  from  various  tools. 

5.5  Communicating  the  results. 

Data  Visualization  is  a  trendy  topic  and  there  are  a  number  of  associated  challenges.  For 
example,  our  graphics  should  highlight  which  factors  contribute  to  the  overall  value  / 
effectiveness  (attributing  cause  and  effect),  but  be  able  to  identify  artificial  relationships  as  well. 
These  might  be  the  result  of  “modelisms”  or  applied  assumptions.  Another  challenge  is  just  the 
sheer  number  of  attributes  our  analysis  usually  needs  to  articulate  simultaneously.  The  disparate 
nature  of  these  attributes  makes  it  difficult  to  visualize  all  at  once.  There  are  a  number  of 
academic  articles  that  use  utility  curves  and  plots  as  a  general  practice  for  displaying  alternatives 
with  all  of  their  associated  disparate  attributes. 

6.  CONTENTIOUS  ELEMENTS  OF  TRADE  SPACE  ANALYSIS 

Each  of  the  following  sub-sections  describe  an  element  associated  with  conducting  analysis  -  not 
necessarily  limited  to  trade  space  analysis  -  that  seemed  to  be  related  to  or  a  source  of  challenge 
for  either  the  conduct  or  communication  of  analysis.  The  intent  of  this  section  is  to  help  analysts 
consider  these  during  methodology  development  or  in  the  documentation  of  CLA.17 

6.1  Ambiguity. 

There  are  a  number  of  sources  for  ambiguity.  Some  of  these  are  related  to  performance  values 
while  others  focus  on  measuring  benefit  (aka  utility  values). 


l7This  is  a  good  opportunity  for  TRAC  analysts  to  share  their  experience  about  when  and  where  assumptions 
must  be  included  to  address  areas  where  NOT  accepting  them  would  invalidate  analysis  as  a  whole. 
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Performance  value.  There  are  only  so  many  computational  resources  available.  “Boundaries  of 
the  input  parameters  and  value  spacing  [had  to  be]  carefully  thought  through... excessive  inputs 
create  so  vast  a  solution  space  that  it  might  be  difficult  or  impossible  to  make  analytical  insights” 
(Hong,  Wee,  &  Kiat).  To  counter  this,  one  suggestion  is  to  set  a  minimum  and  maximum  value 
for  the  variable  and  then  a  “most  likely  value”.  As  a  friendly  reminder,  it  is  important  that  there 
is  an  associated  study  assumption  that  addresses  the  selected  set  of  parameter  values. 

Design  of  Experiment18  is  strongly  recommended  to  minimize,  while  justifying,  the  number  of 
runs  required.  The  GCV  MSB  AoA  Phase  1  conducted  a  parametric  analysis  of  the  vehicle 
attributes  using  a  fractional  factorial  DoE  to  combat  time  and  resource  constraints.  They  looked 
at  four  vehicle  factors  (gun,  hit  avoidance,  sensors,  and  vehicle)  at  two  levels  (base  and 
improved). 

Another  note  on  performance  values:  depending  on  the  maturity  or  nature  of  the  attributes  being 
assessed,  rough  order  magnitude  estimations  are  required.  One  example  provided  from  the  CPR 
work  was  cost:  “trying  to  get  cost  figures  for  solutions  that  were  /  are  nothing  more  than 
initiatives  with  no  real  definitions,  or  cost  figures  for  solutions  that  are  not  materiel  -  like  costing 
for  training  a  BCT’s  worth  of  Ranger  school  graduates”19. 

Utility  value 20.  It  is  a  “dimensionless  parameter  that  reflects  the  ‘perceived  value  under 
uncertainty’  of  an  attribute.  A  utility  function  is  typically  yields  a  value  from  least  desirable,  0, 
to  most  desirable,  1 .  There  are  a  number  of  benefits  and  drawbacks  to  applying  Utility  Theory. 
However,  utility  values  enable  the  application  of  a  unit-less  measure  to  a  number  of  disparate 
factors  that  is  representative  of  benefit.  This  reduces  the  number  of  dimensions  required  to 
communicate  the  specific  perfonnance  of  a  set  of  disparate  parameters. 

6.2  Uncertainty  vs.  Risk. 

Uncertainty  is  a  related  term  to  ambiguity.  “Oftentimes  the  uncertainty  inherent  in  system 
development  is  treated  synonymously  with  risk  and  as  such  carries  negative  connotations. 
(Hastings  and  McManus,  2003)  describes  a  more  generalized  framework  for  thinking  about 
uncertainty,  highlighting  the  fact  that  there  is  both  upside  as  well  as  downside  risk  associated 
with  uncertainty...”  (Ross  &  Hastings,  The  Trade  Space  Exploration  Paradigm,  2005).  In  terms 
of  trade  space  representations  there  are  a  number  of  approaches  for  including  such  uncertainty. 
The  article  points  to  another  reference  (Walton,  Myles.  Managing  Uncertainty  in  Space  Systems 
Conceptual  Design  Using  Portfolio  Theory.  PhD,  Massachusetts  Institute  of  Technology,  2002) 
which  explores  using:  error  ellipses,  best-worst  case  lines,  and  color-coding  to  represent 
tradespace  uncertainties,  as  well  as  an  uncertainty-value  plot.  Consider  the  levels  of  risk 


18  You  can  find  an  Overview  of  DoE  on  the  TRAC  SharePoint  COBP. 

19  Mr.  Larimer,  Principle  Analyst  13Marl5  Email. 

20  “The  Tradespace  Exploration  Paradigm”  article  by  A.  Ross  and  D.  Hastings  from  MIT,  articulates  a  process 
of  value-focused,  broad  trade  space  exploration  that  provides  some  valuable  insights  into  communicating  and 
quantifying  the  impact  of  changing  requirements,  and  uncertainty,  along  with  other  system  properties.  This,  along 
with  several  related  articles  are  available  on  the  Trade  Space  Community  Page. 
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adversity  a  decision  maker  may  want  to  explore  in  terms  of  cost,  schedule,  and  or  technology 
risk. 


6.3  Sphere  of  Influence. 

Regardless  of  the  topic,  there  is  only  so  much  that  is  in  our  sphere  of  influence. 

Example  related  to  the  data  used  to  run  analysis:  The  utility  values  used  to  run  the  IBCT  /  ABCT 
/  SBCT  CPR  optimization  model  were  collected  using  the  CNA  SME  elicitation  methodology. 
TRAC  did  not  create  those  values,  nor  were  they  100%  privy  to  the  details  of  execution  of  the 
elicitation. 

Example  related  to  the  analysis:  The  Abrams  ECPII  C-BA  assessed  the  benefits  of  a  set  of 
technologies  to  the  Abrams  platform,  established  if  the  technologies  could  be  implemented  as  an 
ECP,  and  developed  COA  to  cost-effectively  modernize  the  tank.  The  analysis  evaluated  nine 
technologies,  some  of  which  were  not  necessarily  under  program  control  (e.g  FLIR). 

6.4  Noise. 

Depending  on  the  type  of  trades  being  assessed,  there  is  a  certain  amount  of  “noise”  that  must  be 
accounted  for.  Noise  factors  can  be  inflicted  by  the  operational  environment,  such  as  enemy 
behavior  or  weather  conditions.  Others  are  directly  related  to  the  physical  components  of  the 
system,  such  as  failure  rates,  especially  at  early  stages  of  concept  development.  Some  of  these 
considerations  are  accounted  for  in  our  modeling,  others  are  not. 

7.  SUMMARY 

On  a  conceptual  level,  a  successful  approach  to  trades  analysis  must  include  (1)  an  approach  that 
provides  transparency  of  assumptions  and  constraints,  (2)  a  method  to  simultaneously  evaluate 
the  costs  and  benefit  of  decision  not  often  measured  in  dollars,  and  (3)  the  means  to  examine 
broader  sets  of  alternatives  over  multiple  scenarios  and  to  allow  decision  makers  to  visualize  and 
interest  with  the  data  that  supports  their  decisions  (Kane,  MAJ  GEN  USAF  &  Bartolomei,  PhD 
LT  COL  USAF,  March- April  2013). 

With  regard  to  the  execution  of  trade  space  analysis,  it  spans  three  dimensions:  physical 
perfonnance,  operational  capability,  and  value  or  utility.  These  have  unique  measures,  but 
should  use  the  same  input  variables.  Trade  space  analysis  concepts  pertain  to  any  combination 
of  dimensions.  This  guide  focused  describing  elements  of  trade  space  analysis  as  it  relates  to 
TRAC  work.  It  does  not  prescribe  methods,  but  hopefully  provides  enough  detail  to  generate 
ideas.  It  links  to  numerous  resources  (some  TRAC  had  already  created  on  method  descriptions); 
codes  of  practice,  texts,  and  articles  related  to  specific  tools  and  techniques,  etc. 

This  document  is  supplemented  by  the  contents  of  the  Trade  Space  Community  of  Practice 
page21:  articles,  briefs,  links  to  TRAC  products,  and  areas  for  analysts  to  collaborate  on  the  topic 


21  APPENDIX  B.  COMMUNITY  OUTLINE  is  an  overview  of  the  Community  Site  contents. 
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and  share  their  experience.  This  is  intended  to  be  a  living  document.  Reference  any  part  of  this 
guide  as  a  discussion  topic  for  recommendations  for  further  revisions  or  topics  to  cover. 

Additionally,  given  the  short  suspense  of  this  project  and  the  large  scope  of  the  topic,  a  number 
of  the  topics  discussed  herein  are  missing  a  visual  example  (e.g.  chart  types,  multiple 
dimensional  plots,  etc.)  It  would  be  valuable  if  others  can  submit  examples  for  the  guide  to 
reference.  It  would  also  be  valuable  if  each  of  the  articles  on  the  community  site  had  an  abstract 
summarizing  the  article  and  identifying  its  relevance  to  the  TRAC  trades  community. 
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APPENDIX  A.  DOD  EXEMPLARS22 


Each  of  these  tools  combines  capability,  performance,  and  value  measures  into  the  tool  structure. 
FACT  (Framework  Assessing  Cost  Technology)  Model 

Summary  of  the  FACT  Overview  Brief.  This  is  a  model  built  for  the  Marine  Corps 
System  Command  in  cooperation  with  Georgia  Tech  Research  Institute.  The  model’s  foundation 
is  Systems  of  Systems  Engineering  concepts.  It  leverages  SysML  software  and  accepted  systems 
engineering  standards  to  incorporate  design  parameter  trades  including  performance,  reliability, 
and  cost  of  a  system  design.  Its  purpose  is  to  understand  interactions  and  identify  implications; 
addressing  elements  like  cost,  reliability,  maintainability,  and  availability.  It  is  advertised  as  a 
decision  support  tool  to  manage  decision  consequences  and  conduct  risk  management. 
Numerous  performance  models  are  accessed  using  metadata  and  cost  is  integrated  using  a  trusted 
O&S  cost  model.  Confidence  analysis  is  also  included.  Subject  matter  expert  distributions  can 
be  applied  to  variables  of  interest;  there  are  probability  and  cumulative  distribution  functions  that 
quantify  uncertainty  of  reaching  thresholds  and  objectives.  It  is  a  browser  based  tool  that  enables 
analysis  as  a  web-service  for  system  design  from  the  component  level  to  a  composite  design. 
There  is  ongoing  work  to  integrate  a  second  research  effort  being  led  by  ARL  to  develop  an 
interface  between  FACT  and  the  Executable  Architecture  Systems  Engineering  (EASE)  model 
that  links  system  concepts  to  combat  simulation.  (Gaughan  &  Metevier,  2014)  Input  would  be 
system  designs  from  FACT.  EASE  model  output  would  go  back  to  FACT  as  measures  of 
effectiveness. 

WSTAT  (Whole  systems  Trades  Analysis  Tool) 

PEO  GCS  worked  with  Sandia  National  Labs  for  the  tool’s  development  along  with 
ARDEC  and  Booz  Allen.  The  tool  has  two  purposes:  “model  the  relationship  between  design 
decisions  and  stakeholder  value  in  order  to  inform  and  potentially  influence  requirements 
documents”  and  “associated  specifications  and  conduct  cost  informed  trades  analysis  based  on 
holistic  design  choices,  while  understanding  the  opportunity  cost  of  each  choice.”  The  model 
integrates  separate  system  models  into  a  single  view.  It  identifies  five  elements  of  Stakeholder 
value:  performance,  acquisition  cost  (unit  cost),  time  to  complete  (schedule  risk),  O&S  cost, 
Spiral  upgrades  (growth  potential  or  long  term  viability).  The  whole  system  trade  analysis 
process  starts  with  understanding  requirements  and  functional  objectives,  mapping  requirements 
to  those  functional  objectives,  then  identify  and  map  to  a  product  structure,  then  the  technology 
options  and  metrics  for  evaluating  those  technology  options  can  be  integrated.  The  application 
of  value  functions  and  priority  weightings  provides  the  last  amount  of  context  for  the  graphical 
output  of  the  model.  The  model  uses  a  genetic  algorithm  evaluating  generated  alternatives. 
(Edwards  (PEO  GCS),  Lawton  (Sandia  Nat' 1  Labs),  Cilli  (ARDEC),  &  Peterson  (BA)) 

CP  AT  (Capability  Portfolio  Analysis  Tool) 


22  Each  title  is  a  link. 
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Summary  of  sections  1.2  and  1.3  of  the  CPAT  V&V  Report:  CP  AT  is  an  acquisition  tool 
designed  to  identify  the  optimum  courses  of  action  (cost,  schedule  and  perfonnance)  for  portfolio 
investment.  Again,  PEO  GCS  worked  with  Sandia  National  Labs  for  the  tool’s  development. 
The  tool  was  designed  to  complement  the  formal  AoA  process  and  support  the  transformation 
and  modernization  of  the  combat  vehicle  fleet.  The  perfonnance  model  detennines  a  value  for  47 
vehicles  across  20  roles  using  49  attributes.  However,  not  all  vehicles  are  considered  for  all 
roles.  The  performance  attributes  were  selected  using  requirements  documents,  SME  panel,  and 
input  from  AMSAA.  The  results  of  the  performance  evaluation  are  combined  with  the  cost 
analysis  and  schedule  analysis  as  an  optimization  model  within  CPAT.  It  is  designed  to  provide 
the  ability  to  understand  the  trade  space  between  cost,  schedule,  and  perfonnance  to  assist  in 
planning  for  the  overall  Combat  Vehicle  portfolio  and  fleet  modernization.  CPAT’s  intended 
use:  to  identify  COA  for  the  PEO  GCS  portfolio  investment,  provide  analytical  underpinnings 
that  support  an  achievable  and  affordability  Combat  Vehicle  Modernization  Strategy,  act  as  a 
decision-making  tool  to  provide  rapid  value  assessment  for  alternative  related  questions  (support 
the  user  community  in  requirements  development  for  CBA-type  analysis).  Long  term  objectives: 
annual  updates,  support  future  investment  decisions,  what-if  scenarios,  integrate  updated  data  to 
support  PM’s  technology  trade  assessments.  (Dell,  Ewing,  MacCahnan,  &  Whitney,  2013)  The 
V&V  report  provides  insight  into  the  mechanics  of  the  tool  along  with  some  limitations  and 
recommendations  for  improvement. 
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APPENDIX  B.  COMMUNITY  OUTLINE 


Status  -  50%  Complete.  Site  has  been  established.  Needs:  content  to  be  organized,  public 
notification  that  the  site  is  available  for  use,  guide  products  uploaded,  permissions  set  to  TRAC 
ALL  to  be  able  to  contribute,  add,  delete,  etc. 

Mission/Purpose  of  the  Trade  Space  Community: 

The  page  is  an  open  forum  for  anyone  to  contribute  resources,  post  questions,  post 
answers/responses,  revise/refine  content,  collaborate  on  any  topic  related  to  TRAC  trades  studies 
or  Trade  Space  Methodologies  in  general.  The  objective  is  to  continue  to  expand  TRAC  state  of 
the  art  in  conducting  trades  analysis,  provide  a  forum  to  leverage  the  collective  experience  of 
TRAC  analysts. 

Member  List:  Self  nominated.  In  the  spirit  of  knowledge  sharing,  if  you  put  your  name  on  the 
list,  please  make  a  commitment  to  visit  it  at  a  regular  interval  with  a  goal  to  contribute  a 
discussion  topic/question,  comment,  link,  article,  etc.  during  each  visit.  There  is  a  link  to 
instructions  on  how  to  set  up  an  alert  on  the  page. 

Identified  Users  (those  who  might  benefit  /  participate  in  the  community  and  why) 

Analyst  -  PD,  literature  search,  find  others  to  collaborate 

Study  Lead  -  Identify  potential  SME  and  resources,  methodologies,  etc. 

Discussion  Threads:  These  are  enduring  threads  and  should  group  the  majority  of  discussion 
topics.  However,  please  feel  free  to  add  others. 

Definitions 

Methodologies 

Trades  and  DOTMLPF-P 

Challenges 

Lessons  Learned 

Analyst  Guide 

Resources  -  Desired  end  state:  include  a  link  to  the  document,  and  a  brief  abstract  for  each. 
Prefer  minimal  uploads. 

-  Study  Links  (TRAC) 

Article  Links  (Academic,  DoD) 

Website  Links 

o  Innovation  /  Things  to  try  (Data  Visualization,  QFD  model,  etc.) 
o  Other  online  communities  (milSuite?) 

Note:  there  is  no  activated  calendar  associated  with  this  community.  Potential  uses,  if  the  group 
decides,  would  be:  anyone  may  schedule  events  for  the  group  (Focused  PD  topics, 
brainstorming  sessions,  collaborative  meetings)  or  advertise  when  study  trade  workshops 
are  being  held,  etc. 
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APPENDIX  C.  QFD 


The  following  is  a  NOTIONAL  example  of  the  application  of  QFD.  Figure  6:  Notional 
QFD  model  1  -  Capabilities  and  Performance  Attributes  maps  the  relationship  between  the  desired 
capability  requirements  of  a  long  range  precision  munition  and  the  performance  characteristics 
that  enable  them.  The  matrix  provides  the  ability  to  visually  map  characteristics  of  capability 
needs  to  perfonnance  requirements.  The  right  side  of  the  table  below  shows  exemplar 
operational  capabilities  of  a  long  range  precision  munition.  The  elements  at  the  top  of  the  table 
describe  the  requisite  perfonnance  characteristics  to  enable  the  desired  capabilities.  The 
definitions  help  identify  appropriate  measures  (MOE  and  MOP).  The  body  of  the  table  identifies 
the  relationships  between  perfonnance  characteristics  the  capabilities.  The  triangular  section  at 
the  top  identifies  any  interaction  between  performance  parameters.  In  QDF,  these  areas  are 
codified  to  articulate  the  extent  of  the  relationships.  In  the  example  below  Upper  case  X’s 
identify  the  most  significant  contributors  to  the  capabilities  and  the  +  /  -  in  the  top  section 
articulates  the  nature  of  the  interrelationship  of  two  attributes.  Consider  the  desired  capability  of 
being  able  to  service  targets  from  a  standoff  position;  the  principle  performance  characteristic  is 
an  extended  range  munition.  The  framework  provides  the  ability  to  map  that  directly  to  a 
threshold  level  of  400km.  The  upper  portion  of  the  map  suggests  there  is  a  negative  relationship 
between  being  able  to  provide  an  extended  range  and  remain  IM  compliant.  Additionally,  it 
shows  that  there  is  a  positive  relationship  between  an  extended  range  and  having  a  responsive 
system. 

j  X  Primary  Contributor 

•  X  Contributes 

!  ! 

|  +  Positive  Relationship  ; 

i  -  Negative  Relationship  j 


Figure  6:  Notional  QFD  model  1  -  Capabilities  and  Performance  Attributes 
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Depending  on  the  scope  of  the  analysis,  the  same  framework  can  map  performance  requirements 
to  specific  enabling  technologies.  Figure  7:  Notional  QFD  model  2  -  Performance  Attributes  and 
Enabling  Technologies  addresses  another  set  of  relationships,  directly  mapping  back  to  the  original 
capability  requirements.  The  same  simple  codification  is  used  to  describe  the  relationships 
between  each  of  the  characteristics  and  the  interactions  between  technologies.  Now  consider  the 
perfonnance  attribute  of  extended  range;  it  is  enabled  by  having  a  low  warhead  weight.  Having 
a  low  warhead  weight  might  have  a  significantly  positive  relationship  with  being  able  to  house 
more  than  one  munition  per  pod. 
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Figure  7:  Notional  QFD  model  2  -  Performance  Attributes  and  Enabling  Technologies 

The  areas  at  the  bottom  and  to  the  right  of  the  table  provide  the  ability  to  communicate  a  number 
of  other  relationships.  The  area  to  the  right  of  the  chart  could  be  used  to  conduct  a  comparative 
analysis  of  other  technologies  or  options  to  achieve  the  desired  capabilities.  The  area  at  the 
bottom  could  be  used  to  identify  target  values,  articulate  which  attributes  contribute  the  highest 
cost,  are  technologically  difficult  to  achieve,  pose  the  greatest  risk,  or  to  conduct  comparative 
analysis  of  a  number  of  alternatives’  perfonnance.  Additionally,  priorities  and  importance 
weights  can  be  included  for  both  perfonnance  characteristics  and  capability  attributes  which  may 
help  illuminate  the  more  “critical”  elements  /  decisions. 
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